




<!doctype html public "-//IETF//DTD HTML//EN">
<html>

<head>

<title>
CPS710
Assignment 1
</title>

<LINK REL=StyleSheet HREF="/~cps710/CSS/h.css" TYPE="text/css" MEDIA=screen>
<LINK REL=StyleSheet HREF="/~cps710/CSS/main.css" TYPE="text/css" MEDIA=screen>
<LINK REL=StyleSheet HREF="/~cps710/CSS/theme.css" TYPE="text/css" MEDIA=screen>

</head>




<body>

<center> <table noborder width=100%>

<tr>

<td align=left valign=top width=125>
<h1><a href="/~cps710/" target="_top" class="NOUNDERLINE"> CPS710 </a></h1>
</td>

<td align=center valign=top>
<h1>

Assignment 1
</h1>

<td align=center valign=top width=125>
<a href="http://www.ryerson.ca"
	onmouseover="parent.window.status='General Information about Ryerson'; return true" >
	<IMG SRC="/~cps710/Pictures/Ryerson2linesMedium.gif" ALT="Ryerson University" border=0></a>
	</a>

</tr>

<tr>
<td align=center valign=top width=125>

</td>

<td align="center" valign="top">

<h2>Scanner</h2>


</td>

<td align=center valign=top width=125>

<h4>Due: October 6</h4>

</td>



</tr></table></center>

<hr>


<h3>Revised assignment</h3>
Please note that this assignment was modified at 2:00AM on September 25th.
Changes are described in a separate section at the bottom of this page.

<h3>Objectives</h3>
In this course you will be developing an interpretor for a high-level
language, called HL, which manipulates sets and strings.
<p>
In this assignment you will be using JavaCC to build HL's scanner
and working on HL's error manager.
<p>
This should familiarize you with the use of compiler generators
and the scanning process.

<h3>Preliminary Work</h3>
Before you start your assignment,
you will need to set up the environment under which you will be working.
You can develop your assignment on whatever platform you prefer.
However, we will only test it only on one computer: elara@scs.ryerson.ca,
so we recommend strongly that you verify your work on this platform before submitting it.
In all cases, you will need to do the following:
<ul>
<li>Create a directory where you will do your work.
<p><li>Copy the 3 following files to that directory:
<ul>
<li>You can start your work with this skeleton version of
<a href="Scanner.jj">Scanner.jj</a>.
<li>This <a href="makefile">Makefile</a> will be used to compile your program.
You can learn about the <a href="http://www.gnu.org/software/make/">make program</a>
by typing "man make" on elara or from the
<a href="http://www.gnu.org/software/make/manual/">GNU Make Manual</a>.
<li>The makefile will link your assignment with
<a href="TestScanner.java">TestScanner.java</a>.
This program is a modification of TokenTest.java provided in the textbook.
Do not modify this file as we will be using the one posted here.
</ul>

<p><li>If you are developing in UNIX on the command line,
you can use the makefile to recompile and test your scanner.
You will need to modify it though when you add the two new classes.
If you are working in an IDE, you will need to set it up yourself.
You may want to look at the makefile for the sequence of actions and flags.
<p><li>Compile everything (make will do this if you are working in UNIX on the command line) once to make sure that your compilation settings are correct.  This will also generate additional files that you can modify later
</ul>

<h3>Assignment</h3>
Once you have set up the environment, and verified that it runs smoothly,
you can start your <a href="description.pdf">assignment</a>.

<h3>References</h3>

<h4>Starting points</h4>
The following are good places to get started so you can produce
a very basic JavaCC file that detects the simple tokens and skips whitespace.
<ul>
<li>
<a href="http://www.cs.lth.se/Education/LTH/EDA200/EDA200.2001/Labbar/L1/javaccscanning.html">
Scanning in JavaCC</a> is a good place to start:
The first 2 links (example and quick guide) provide an excellent description
of how to produce a simple scanner with JavaCC.
Unfortunately the other links are broken.

<li>Sections 2.4 and 2.5 of course textbook are quite thorough.

<li>Wikibooks'
<a href="http://en.wikibooks.org/wiki/Compiler_Construction/Using_JavaCC%27s_TokenManager">
Compiler Construction/Using JavaCC's TokenManager</a> also gives a
good overview.
The comment section will get you properly started with comments.

</ul>

<h4>Advanced Topics</h4>
<ul>

<li><a href="https://javacc.dev.java.net/doc/javaccgrm.html#prod29">
JavaCC Grammar - complex_regular_expression_choices</a>
contains a BNF description of the grammar of the regular expressions
that can be used to describe tokens and other items in the lexer.

<li>Sections 2.4 and 2.5 of course textbook contains the best explanation
I have seen so far of how to handle comments.

<li><a href="https://javacc.dev.java.net/doc/tokenmanager.html">
JavaCC [tm]: TokenManager MiniTutorial</a>
gives a good overview of how JavaCC handles tokens and how to write
a JavaCC program.

<li><a href="https://javacc.dev.java.net/doc/errorrecovery.html">
JavaCC [tm]: Error Reporting and Recovery</a>
is also helpful for this asignment.

<li><a href="https://javacc.dev.java.net/doc/apiroutines.html">
JavaCC [tm]: API Routines</a>
contains a comprehensive list of all classes, methods,
and variables available for use by a JavaCC user.  

<li><a href="http://www.idevelopment.info/data/Programming/java/JavaCC/The_JavaCC_FAQ.htm">
The JavaCC FAQ</a>
<li>See <a href="../references.shtml">course references</a> for further JavaCC references.
<li><a href="http://forum.java.sun.com/thread.jspa?threadID=537957&messageID=2649432">
	"Unsafe vector ..." bug </a> you may encounter.  The fix is described in this link
	and implemented in the makefile.
</ul>

<h3>How to submit the assignment</h3>
<ul>
<li><a href="../submission.shtml">Submit electronically</a> Scanner.jj, Token.java, CharToken.java,
NumberToken.java, and TokenMgrError.java
<li>Print, fill out, and hand in the marking sheet (
<a href="markingA1.doc">word format</a>
or
<a href="markingA1.pdf">pdf format</a>
).
</ul>

<hr>
<h3>September 25th Assignment Changes</h3>

<h4>Assignment description:</h4>
<ul>
<li>There have been a few corrections to simple tokens:
<ul>
<li>The "string" keyword was changed to "str" so as not to clash with the constant STRING token.
<li>Similarly the "CHAR" constant token was renamed "CHARACTER" so as not to clash with the "char"
keyword.
<li>The SEMICOL token matches string ";"
<li>Two new keywords have been added: "print" and "println"
</ul>
<li>The definition of CHARACTER and STRING tokens has been clarified, with the valid characters
listed.
<li>Comments should be declared as SPECIAL_TOKEN instead of simply removed.
<li>Section 2 on changes to Token class and its 2 subclasses has been changed substantially.
Please look at it carefully.
</ul>

</ul>

<h4>File changes</h4>
<ul>
<li>The printing routines of TestScanner.java have been improved.
<b>Please download this file again.</b>
<li>The References section of this page has been reworked and beefed up.
</ul>

<p><hr>
<center>

This page is maintained by
Sophie Quigley
(<a href="mailto:cps710@scs.ryerson.ca">cps710@scs.ryerson.ca</a>)


<br>
Last modified 
Monday, 25-Sep-2006 02:13:41 EDT

</center>

</body>
</html>




